Scott Watson‘s White Knight Feedback, etc on ECHOMAC
December 1, 1989
It's an unfortunate fact that my present workload hasn't allowed me to be an active participant in EchoMac, but I would like to assure everyone that your comments, criticisms, insults, suggestions, etc. are filtering back to me, and I do appreciate them (yes, even the insults). I'm going to attempt to set up an EchoMac node here locally in the near future so that I can be more in touch with this important network and take advantage of the knowledge and experience here.
After looking through a 91K text file of recent EchoMac messages, it's pretty obvious that a few threads have gotten unnecessarily ugly when a simple message from me could have prevented it. I'll try (belatedly, I concede) to do that now, and in the messages to follow.
On the subject of the "White Knight" name, let me assure anyone who has doubt that I am neither Aryan nor a supremicist. I am, in fact, of Scot/Irish/American Indian descent which would make me one of the "mongrel slave races" that Hitler intended to subjugate.
To quote Groucho Marx, "I would never belong to a club that would have me as a member", excepting, perhaps, the Beaver County Gun, Rod, and Bloody Mary Society). I am not, nor ever have been a "resident of the Southeastern U.S.", and don't keep abreast of which activists, lunatics, desperados, or criminals have decided to call themselves "White Knight".
If I had known, I doubt that it would have affected my decision to use that name any more than knowing that Charles Manson has on occasion referred to himself as Jesus Christ would cause me to change my religion.
If you are looking for a racist in my pajamas, or a horrible, mystic, tabloid-worthy secret in the name "White Knight", methinks you fish for trout in strange lakes.
It wouldn't make any sense to explain why I chose the name "White Knight". The point is that the name is pointless.
By now, I'm sure that the benchmarks I released previously have made their way through the loop. There has been some discussion on why I benchmarked through a direct serial cable rather than in what has been termed a "real world" scenario through modems and voice grade telephone lines. I will explain.
When you benchmark, it's important to qualify what it is that you are attempting to measure. In my benchmarks, I tried to measure four different factors: VT100 emulation speed, straight text scrolling speed, XMODEM file receive speed, and ZMODEM file receive speed. There are a _lot_ of things I didn't measure in these benchmarks, but these are the things I felt were most significant and additively noticeable to a communications program user. It's very important that I state that these benchmarks were never originally designed to be released to the public. They weren't done as public relation gizmos, but rather as to tell me where I stood during development. It would have done me no good to lie to myself.
Notice that what I am attempting to measure is the efficiency of the implementation of these features, not the overall efficiency of the emulation or file transfer protocol itself as to moving data - these things would remain constant and would be different benchmarks to run (if we were so inclined).
An assumption I made is that the emulation and file transfer protocols were implemented correctly (nothing in my testing proved otherwise), therefore there is no reason to introduce consistent noise bursts during the measurement - at that point, I would be measuring the protocol's efficiency to error-detect/ correct, rather than the efficiency of the implementation of the protocol. The error recovery logic, tolerance to timeouts, etc. are cast in stone by the protocol specifications, if someone wanted to introduce errors to confirm that a given program conforms to the protocol specification, that's fine. But that's not a benchmark, that's an affirmation of conformance to a standard.
The reason I say this is that the most time consuming parts of a file transfer protocol are disk I/O and error detection. Since disk I/O doesn't happen when receiving faulty blocks, forcing errors wouldn't test the implementation efficiency of this. Yes, disk I/O implementation does have a great effect on the overall efficiency of the implementation. Try doing 10000 1 byte writes versus 1 10000 byte write to check me on this.
When doing error checking, my research showed that the largest number of errors happened as bursts of noise in the middle of a block.
Clearly, the same error-detecting routine has to be done with a good block as with a bad block. It takes the same amount of time to make the decision "it's good" or "it's bad". To tell the other side it's bad takes only a small number of bytes, and cleverness of implementation won't be significant there. Error recovery has as much to do as when the error is detected and reported, as it does for the sender to respond and reply with a retransmission. In other words, it's nonsense to benchmark worst possible circumstances because they can't be easily duplicated, nor are they an indication of what the typical user will experience.
There was no reason to even attempt to time error-recovery implementation efficiency, because first of all that would take the sort of equipment I don't have access to (the burst would have to be precisely consistant and precisely timed in the data stream), and second, because having had my hands dirtied in the guts of various protocols, I can say with a certain level of expertise that the timeout recovery and error-detection/correction routines are not where the greatest amount of time is spent in a protocol. It's my observation, for instance, that ZTerm could speed up it's XMODEM times by buffering blocks and doing larger writes, and by not doing so frequent writing to its status window.
Whether I'm right or wrong (I'm sure Dave will let me know (grin)), it's pretty obvious that neither of these things have anything to do with the protocol specifications.
Someone made the comment that it's easy to fine-tune a protocol to run like greased lightening in a direct serial line situation, but that has no bearing how the program will perform in the "real world". It just ain't so. If one program has X efficiency rating when running through a direct serial link, and a second program has Y efficiency, the relationship of X to Y will remain proportional when both are operating in the "real world" under _precisely_ similar circumstances (be they good or bad circumstances).
As an aside, I'd be interested to know just how a person would set out to implement a protocol (whose function is to respond to errors) that did extremely well under circumstances where errors can't occur. Seems awfully superfluous to me, and certainly not something I'd want to have on the market.
At any rate, let it be known that there are no prizes given for the fastest protocol implementation to run on a direct cable connection, and no one who would buy a product (I hope) based on the benchmarks I released. When a person chooses a communications program, it is for very personal reasons that I wouldn't want to debate. There's a helluva lot more to "which program is the one for me" than looking at numbers like the ones Bob Nordling & I have uploaded.
Conceivably, there is a program that operates only half as fast in the benchmarks I did, but is a much better overall program for a given individual due to specific features not quantified in these tests. If that's the case, then a person would be a fool to choose a program based on those numbers.
My first paragraphs describe what is a "benchmark". The last paragraph is the only qualification for "real world" testing - what feels most comfortable, and offers the greatest benefits in proportion to drawbacks to the individual. Benchmarks, properly done, are good for a limited comparison, and I paid a hell of lot of attention to them during development of White Knight. But they are by no means conclusive proof of superiority.
S.W.
--- Tabby 2.1
* Origin: MacInfo BBS - Newark, CA. @415-795-8862 (HST)RRH (1:204/555)
